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PERSONALIZED SUBTITLE SYSTEM 

5 BACKGROUND OF THE INVENTION 

This invention pertains generally to providing subtitles 
and more specifically to providing subtitles personalized for 
a user. 

Since the emergence of consumer digital media, such as 
10 the^Compact Disc (CD) introduced in the 1980' s, the trend 
towards -digitization of media has continued unabated. The 
Digital Video^ Disk (DVD) platform has experienced an 
unprecedented "/adoption rate and the digital production, 
formatting, distribution and archiving of a vast majority of 
15 all media is likely to accelerate. Concurrently, the advent 
of broadband Internet services, digital television, digital 
t . streaming media, and a myriad of digital player devices „. have 
increased the viability for the commercial distribution of 
media without physical packaging. The emerging trend is 

2 0 towards streaming digital channels 4 and the building of 

Personal Media Libraries and away from analog broadcast 
consumption. Predictions of the eventual obsolescence of the 
CD and DVD have begun to surface. 

While all of these factors, along with a general 

25 globalization trend, have served to increase the distribution 
and availability of media, the potential for providing 
universal access, and adding value and enhanced content to 
media has yet to be realized. For example, the text 
associated with media, i.e. subtitles and lyrics, are either 

30 available in only limited fashion, or not available at all. 

If translation via subtitles were more readily available to 
accompany associated media, a larger global audience would be 
able to enjoy content from sources all around the world. 
Additionally, an archival system containing the subtitles, 

35 lyrics, and transcripts of media will allow for new forms of 
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content search and the emergence of new educational and 
commercial opportunities enabled by such a feature. 

5 Therefore a need exists for computer programs, systems, 

and protocols that allow the archiving and delivery of 
(personally selectable) subtitles and other complementary data 
to media files for which there is a meaningful language 
component. Furthermore, a need exists to make such subtitles 

10 and other complementary data available both for media which 
has been downloaded and archived as well as media which is 
being downloaded, i.e. experienced live or streamed, whether 
from a pre-recorded event, or from an actual real time event, 
personalized to the individual experiencing the media. 

15 Various aspects of the present invention meet such needs. 

SUMMARY. OF THE INVENTION 

In one aspect of the invention, a personalized subtitle 
system includes a Heads Up Display (HUD) worn by a user in a ! 
20 public venue such as a movie, theater, playhouse, or stadium. 

The HUD is coupled to a personalized subtitle system 
controller that the user utilizes to control the operations of 
the personalized subtitle system. The user utilizes the HUD 
and personalized subtitle system controller to select and read 

2 5 captioning or subtitle information for a public event such as 

a movie, play, or sporting event. In this way, subtitles in a 
variety of languages can be supplied for the public event. 

In another aspect of the invention, useful when a user is 
viewing pre- formatted presentations such as a movie in a 
30 theater, the subtitles are stored by the personalized subtitle 
system. Synchronization signals are transmitted by a cinema 
server to the personalized subtitle system controller via a 
wireless communications network in order to synchronize the 
pre-stored subtitles with presentation content. 

35 
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In another aspect of the invention, the user provides a 
synchronization signal to the personalized subtitle system 

5 controller in order to synchronized pre-stored subtitles with 

presentation content. 

In another aspect of the invention, the subtitles are 
transmitted as needed to the personalized subtitle system 
controller via a wireless communications network. 

10 In another aspect of the invention, the personalized 

subtitle system is used in conjunction with a conventional 
subtitle display system, such as a DVD player for home use. 

In another aspect of the invention, a user access a p 
subtitle server via a communications network to obtain 

15 subtitles. The subtitle server includes subtitles and 
associated metadata describing the subtitles. The user may 
then use the. metadata in order to determine which subtitles to 
access . 

r 

2 0 BRIEF DESCRIPTION OF THE DRAWINGS 

These and other features, aspects, and advantages of the 
present invention will be more fully understood when 
considered with respect to the following detailed description, 
appended claims, and accompanying drawings, wherein: 
25 FIG. la is a block diagram of a personalized subtitling 

system in accordance with an exemplary embodiment of the 
present invention ; 

FIG. lb is a personalized subtitling system used for a 
cinema in accordance with an exemplary embodiment of the 
30 present invention; 

FIG. 1c is a sequence diagram of a personalized subtitle 
system in accordance with an exemplary embodiment of the 
present invention ; 

35 
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FIG. Id is a sequence diagram of the operation of a 
dynamic configuration process in accordance with an exemplary 
5 embodiment of the present invention; 

FIG. le is a sequence diagram of the operation of a 
cinema server in accordance with an exemplary embodiment of 
the present invention; 

FIG. If is a sequence diagram of the operation of a 
10 cinema server transmitting synchronization signals in 
accordance with an exemplary embodiment of the present 
invention; 

FIG. 2 is a screen display from a personalized subtitle 
system controller in accordance with an exemplary embodiment 
15 of the present invention ; 

FIG. 3 is a block diagram of a personalized subtitling 
system having a separate input device in accordance with an 
exemplary embodiment of the present invention; 

FIG. 4a is a hardware architecture diagram of a data 
20 processing system suitable for use as a personalized subtitle 
system controller in accordance with an exemplary embodiment 
of the present invention; 

FIG. 4b is a hardware architecture diagram of a data 
processing system suitable for use as a cinema server in 
25 accordance with an exemplary embodiment of the present 
invention; 

FIG. 5 is a block diagram of subtitle to content 
synchronization method wherein the subtitles are associated 
with presented content in accordance with an exemplary 
30 embodiment of the present invention; 

FIG. 6 is process flow diagram of a personalized subtitle 
display process in accordance with the subtitle to content 
association method of FIG. 2; 

FIG. 7 is a block diagram of subtitle to content 
35 synchronization method wherein presented content has an 
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associated synchronization signal in accordance with an 
exemplary embodiment of the present invention; 

5 FIG. 8 is a block diagram of subtitle to content 

synchronization method wherein a user supplies a 
synchronization signal in accordance with an exemplary 
embodiment of the present invention; 

FIG. 9 is process flow diagram of a personalized subtitle 

10 display process in accordance with FIG. 4 and FIG. 5; 

FIG. 10 is a block diagram depicting using the 
personalized subtitle system with a variety of enhanced 
content sources in accordance with an exemplary embodiment of 
the present invention; and 

15 FIG. 11 is a block diagram depicting using the 

personalized subtitle system at a live event in accordance 
with an exemplary embodiment of the present invention. ; 

DETAILED DESCRIPTION 

20 FIG. la is a block diagram depicting a personalized 

subtitle system incorporating a subtitle server in accordance 
with an exemplary embodiment of the present invention. A 
personalized subtitle system accesses a subtitle server 400 in 
order to obtain subtitles for various types of media. The 

25 subtitles are included in files that are herein termed ".sub" 
files in reference to their common three character extension. 
These .sub files may have internal formats that reflect the 
type of media that the .sub file is intended to be used with. 
For example, a .sub file may be a text file associated with 

30 any media to which subtitles are to be added, such as movies, 
television shows, digital video discs, digital music files, 
radio shows, audio books, and other digital mediums. 

The subtitle server may maintain a .sub file database 820 
supplied with .sub files from a variety of sources. For 

35 example, .sub files may be generated and supplied by content 
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publishers 822, individuals 824 who create .sub files for 
altruistic or hobby purposes, and aggregators 826 who collect 
5 .sub files for profit or other purposes. The subtitle server 

may also access more generalized metadata 827 such as data 
about closed captioning, lyrics, transcripts of public events, 
etc . 

The content served by the subtitle server may be searched 
10 in a variety of ways. A user interface 830 provides searching 
by content title, artists or actors, audio tracks, versions, 
dates, directors, original languages, etc. The user interface 
may also allow more sophisticated queries such as allowing 
queries by type of subtitle needed, whether a .sub file, what 
15 language is needed, whether the material is from a closed 
caption or not, etc. A user using a personalized subtitle 
system uses the user interface to request (832) an appropriate 
.sub file which is then transmitted (846) to the personalized 
subtitle system for use. The personalized subtitle system 
20 receives (848) the .sub file and plays or synchronizes the 
.sub file with the associated media under the direction (834) 
of the user. 

The personalized subtitle system may provide subtitles 
for a variety of media types. The personalized subtitle 

25 system may provide subtitles for watching a movie 836 as 
previously described. In addition, other types of "live" 
events may be supported, such as listening to live radio 838, 
watching a live television broadcast 840, viewing or listening 
to a live streaming file 842, live performances 844 at public 

30 venues, etc. 

The personalized subtitle system may also provide 
management services 852 for managing a Personal Media Library 
(PML) including downloaded media files 850. The personalized 
subtitle system may transfer (853) the media files to other 

35 devices. A user utilizes the PML management services of the 
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personalized subtitle system to present the media files such 
as watching videos 854, listening to music 856, listening to 

5 audio books, reading electronic books, etc. 

The subtitle server may be coupled to a wide area 
network, such as the Internet. This allows conventional 
search engines 860 to search and index the content of the 
subtitle server for responding to Internet searches by users 

10 for .sub file content. 

FIG. lb is a block diagram of a personalized subtitle 
system used in a cinema in accordance with an exemplary 
embodiment of the present invention. A personalized subtitle 
system 100 provides subtitles 102 for viewing by a user while 

15 the user is viewing an entertainment production or other 
event, such as a movie 104 displayed in a movie theater. The 
subtitles are displayed on a display device, such as a Heads 
Up Display (HUD) device 106. 

Suitable HUD devices are manufactured by The MicroOptical 

20 Corporation of Westwood, Massachusetts, USA. Such a r HUD 
device is a model The DV-1™ Wireless Digital Viewer mountable 
on eyeglasses or safety eyewear. The HUD device provides a 
monocular color quarter Video Graphics Adapter (VGA) image 
with a pixel format of 320 columns by 240 rows with a color 

25 depth of 12 bits. The DV-1™ displays bitmap graphics and text 
and the two modes can be overlaid. Communication between DV-1™ 
and other devices is achieved by establishing a linkage using 
a proprietary protocol over a Bluetooth™ wireless channel. 
The DV-1™ is battery operated. 

30 The HUD device is coupled via communication link 107 to a 

personalized subtitle system controller 108 that includes 
functions for: controlling the operations of the HUD device; 
receiving subtitles; and receiving user inputs from the user. 
The personalized subtitle system controller may receive 

35 subtitles from a variety of sources. In one embodiment, the 
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personalized subtitle system controller receives subtitles 
from a subtitle server, such as cinema server 110, that also 
5 supplies (112) the visual images and audio portions of the 

movie being viewed by the user. The personalized subtitle 
system controller couples (115) to the cinema server via a 
communication network, such as wireless communications network 
114. 

10 In a personalized subtitle system in accordance with an 

exemplary embodiment of the present invention, the 
personalized subtitle system is coupled to the cinema server 
using a communications network employing the IEEE 802.11 
wireless Ethernet protocol commonly known as "Wi-Fi". The 

15 personalized subtitle system controller is further coupled to 
the HUD device using a wireless communication link using a 
communication protocol such as Bluetooth. 

FIG. lc is a sequence diagram of a personalized subtitle 
system in accordance with an exemplary embodiment of the 

20 present invention. In operation, a user personalized subtitle 
system controller 108 receives a request 151 from a user 150 
to access (152) a cinema server 110 associated with a movie 
that the user is viewing. In response to the request for 
access, the cinema server determines (153) which subtitles to 

2 5 transmit to the controller. The cinema server then gets (154) 
the appropriate subtitles 155, synchronized with the content 
of the movie, and transmits the subtitles to the personalized 
subtitle system controller. The personalized subtitle system 
controller uses the received subtitles to generate (154) 

30 formatted subtitles 156 for transmission to a HUD device 106. 
The HUD device receives the subtitles and uses the subtitles 
to generate 158 a subtitle display 160 for display to the 
user. The user may then view the subtitles and the movie 
simultaneously. 

35 
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FIG. Id is a sequence diagram of the operation of a 
dynamic configuration process in accordance with an exemplary 

5 embodiment of the present invention. In a dynamic 

configuration process, a personalized subtitle system 
controller 108 receives a request 161 from a user 150 to 
access 162 a cinema server 110. The cinema server returns 
event information 163 including a list of movies, what screens 

10 the movies are playing on, what times the movies are showing, 
what subtitles are available for each movie, and what channel 
or port each subtitle will be broadcast on. The controller 
formats the received information and transmits the formatted 
information 164 to a HUD device 106 for display to the user. 

15 The controller receives from the user a selection 168 
indicating the movie and subtitles the user wants to view. 
The controller . then configures (170) itself to receive the 
requested subtitles 172. To do so, the controller may 
transmit a controller registration 171 to the cinema server. 

2 0 The controller ..will receive subtitle packets for the desired 

subtitle by receiving on the appropriate channel or port. 

The controller automatically begins transmitting formatted 

subtitles 174 to the HUD device upon reception of the subtitle 

packets. The HUD device uses the formatted subtitles to 
25 generate a subtitle display 17 6 that is shown to the user 

either on the HUD display or by the controller. 

In one personalized subtitle system in accordance with an 

exemplary embodiment of the present invention, the 

personalized subtitle system retains a default language 
30 setting. Through the use of the default language setting, 

subtitle files would automatically be in the default language 

unless specified. 

FIG. le is a sequence diagram of the operation of a 

cinema server in accordance with an exemplary embodiment of 
35 the present invention. The cinema sever may serve subtitles 
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to more than one personalized subtitle system controller, as 
exemplified by personalized subtitle system controllers 108a 

5 and 108b. The cinema server receives configuration 

information 180 and 182 from the personalized subtitle system 
controllers. The cinema server stores the personalized 
subtitle system controller configuration information. To play 
a movie, the cinema server transmits the visual images and 

10 audio portions 186a of the movie to a projection device 179. 

In addition, the cinema server transmits subtitles, 188a and 
190a, associated with the visual images and audio portions to 
each of the personalized subtitle system controller. The 
process of transmitting visual images and audio portions to 

15 the projection device and transmission of associated subtitles 
is repeated continuously, as represented by visual images and 
audio .portions 186b, subtitles 188b and 190b, -and. ellipses 
191. 

The subtitles may be transmitted in a variety of ways:. 

20 In one personalized subtitle system in accordance with an 
exemplary embodiment of the present invention, the cinema 
server transmits packets that are specifically addressed for 
transmission to a specific personalized subtitle system 
controller on a network. To receive the packets, a 

25 personalized subtitle system controller registers itself with 
the cinema server so that cinema server knows what subtitles 
to transmit to the personalized subtitle system controller and 
what address to send them to. 

In another personalized subtitle system in accordance 

30 with exemplary embodiments of the present invention, the 
cinema server sends out packets addressed to a special group 
address. Personalized subtitle system controllers that are 
interested in this group register to receive the subtitle 
packets addressed to the group when the user chooses a 

35 specific subtitle selection. 
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In another personalized subtitle system in accordance 
with an exemplary embodiment of the present invention, the 
5 cinema server sends out packets intended for transmission to 

all personalized subtitle system controllers on a network. 

In another personalized subtitle system in accordance 
with an exemplary embodiment of the present invention, 
subtitles are assigned to a dedicated destination channel or 
10 port. In this embodiment, the personalized subtitle system 
controller does not need to do any filtering of the subtitles. 

In another personalized subtitle system in accordance 
with an exemplary embodiment of the present invention, all 
subtitles are included in a single data stream addressed to 
15 the same destination channel or port. In this embodiment, the 
personalized subtitle system controller filters the received 
subtitle stream to identify the which portions of the subtitle 
stream includes the desired subtitles. 

In another personalized subtitle system in accordance 

2 0 with an exemplary embodiment of the present invention, the 

cinema server transmits a single subtitle to a personalized 
subtitle system controller over a TCP stream. In this 
embodiment, the personalized subtitle system controller tells 
the cinema server what subtitle to transmit. 

25 FIG. If is a sequence diagram of the operation of a 

cinema server transmitting synchronization signals in 
accordance with an exemplary embodiment of the present 
invention. The cinema sever may serve synchronization signals 
to personalized subtitle system controllers rather than 

30 complete subtitles to more than one personalized subtitle 
system controller, as exemplified by personalized subtitle 
system controllers 108a and 108b. In this embodiment, the 
subtitles are stored by each of the personalized subtitle 
system controllers for display to individual users. In 

35 operation, the cinema server receives configuration 
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information 180 and 182 from the personalized subtitle system 
controllers. The cinema server stores (184) the personalized 

5 subtitle system controller configuration information. To play 

a movie, the cinema server transmits visual images and audio 
portions 186a of the movie to a projection device 179. In 
addition, the cinema server transmits synchronization signals, 
192a and 194a, associated with the visual images and audio 

10 portions to each of the personalized subtitle system 
controller. The process of transmitting visual images and 
audio portions to the projection device and transmission of 
associated subtitles is repeated continuously, as represented 
by visual images and audio portions 186b, synchronization 

15 signals 192b and 194b, and ellipses 195. 

In another personalized subtitle system in accordance 
with an exemplary embodiment . of the present ■, invention, the 
personalized subtitle system controller auto-discovers movies, 
display times, screen locations, and available subtitles when 

20 a user walks into the specific movie seating area or lobby. 

FIG. 2 is a screen display from a personalized subtitle 
system controller in accordance with an exemplary embodiment 
of the present invention. A personalized subtitle system 
controller 108 includes a screen display 700 for display of a 

25 menuing system 701 used by a user to control the operations of 
a personalized subtitle system controller. The menuing system 
includes a "Main Menu" menu 7 02 that offers a user a 
communication network logon selection 704. Once the user 
accesses the communications network, the user may then access 

30 a subtitle server or a cinema server via the communication 
network as previously described. The menuing system further 
includes a "Settings" submenu 706 having a "HUD on/off" 708 
selection for turning the HUD device on or off and a "Settings" 
selection 710 for setting various options of the HUD device. 

35 
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The menuing system further includes a "Subtitles" submenu 
712. The Subtitles submenu includes an "on/off" selection 714 

5 for turning the display of subtitles on and off. The 

Subtitles menu further includes a "Settings" submenu 716 having 
a "Language" selection 718 for selecting which language 
subtitles will be displayed in. The Settings submenu further 
includes a "Position" selection 720 for adjusting the position 

10 of the subtitles displayed by the HUD device. The Settings 
submenu further includes a "Size" selection 722 for setting the 
size of the subtitles displayed by the HUD device. Finally, 
the Settings submenu includes a "Color" selection 724 for 
setting the color of the displayed subtitles and a "Font" 

15 selection 725 for selecting a font for the displayed 
subtitles . 

FIG. 3.. is. a block diagram of _ a personalized subtitle 
system having a separate input device in accordance with an 
exemplary embodiment of the present invention. In this 

20 embodiment, the personalized subtitle system controller 108 
includes separate components that are coupled via short range 
communications links. The user utilizes an input device 910 
coupled to a subtitle receiver and HUD controller 911 via a 
short-range communications link 912 such as a Bluetooth 

25 communications link. In operation, the personalized subtitle 
system controller displays previously described menu 
information 7 00 to the user using the HUD device 106 via a 
communications link 107. In response to the menu information, 
the user utilizes the input device to navigate through the 

30 menu system. 

FIG. 4a is a hardware architecture diagram of a data 
processing system suitable for use as a personalized subtitle 
system controller in accordance with an exemplary embodiment 
of the present invention. A data processing system includes a 

35 processor 1000 operatively coupled via a system bus 1002 to a 
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main memory 1004 and an I/O interface control unit 1006. The 
I/O interface control unit is operatively coupled via an I/O 

5 local bus 1008 to a storage controller 1010. The storage 

controller is operatively coupled to a storage device 1012. 
Computer program instructions 1014 implementing a personalized 
subtitle system are stored on the storage device until the 
processor retrieves the computer program instructions and 

10 stores them in the main memory. The processor then executes 
the computer program instructions stored in the main memory to 
implement a previously described personalized subtitle system 
to display subtitles to a user. 

The personalized subtitle system controller further 

15 includes a display device 1018 coupled to the I/O local bus 
via a display controller 1016. The display device may be 
integral to the subtitle system controller .such as displayr700 
of FIG. 2. The personalized subtitle system controller uses 
the display controller and display device to display portions 

20 of a personalized subtitle system user interface to a user. 

The personalized subtitle system controller further 
includes an input device 1022 coupled to the I/O local bus via 
an input controller 1020. An input device may be integral to 
the subtitle system controller as illustrated by controller 

25 108 of FIG. 2 or may be a separate device, such as input 
device 910 of FIG. 3. A user may use the input device to 
transmit synchronization signals to the personalized subtitle 
system controller as previously described. In addition, the 
user may use the personalized subtitle system controller to 

30 provide user inputs in response to the display portions of the 
user interface generated by the personalized subtitle system 
controller. 

The personalized subtitle system controller further 
includes a HUD interface 1026 coupled to the I/O local bus via 
35 a HUD controller 1024. The personalized subtitle system 
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controller uses the HUD interface to transmit subtitles to the 
HUD device as previously described. In one HUD device in 

5 accordance with an exemplary embodiment of the present 

invention, the HUD device includes a wireless communications 
link for receiving subtitles from the personalized subtitle 
system controller. In this embodiment, the HUD interface 
includes a wireless communications device. In another HUD 

10 device in accordance with an exemplary embodiment of the 
present invention, the HUD interface is directly coupled to 
the personalized subtitle system controller. 

The personalized subtitle system controller further 
includes a network device 1030 coupled to the I/O local bus 

15 via a network controller 1028. The personalized subtitle 
system controller uses the network device to access a 
communications network and. .communicate with various sources of 
subtitles as previously described. 

The personalized subtitle system controller may further 

20 include an audio device 1034 coupled to the I/O. local bus via 
an audio controller 1032. The personalized subtitle system 
controller uses the audio device to present audio information 
to a user as previously described. 

In one personalized subtitle system controller in 

25 accordance with an exemplary embodiment of the present 
invention, the subtitle controller includes subtitles 1015 
stored in the memory storage device. These subtitles are 
displayed to a user in response to synchronization signals 
received by the personalized subtitle system controller. 

30 FIG. 4b is a hardware architecture diagram of a data 

processing system suitable for use as a subtitle server in 
accordance with an exemplary embodiment of the present 
invention. A data processing system includes a processor 12 00 
operatively coupled via a system bus 12 02 to a main memory 

35 1204 and an I/O interface control unit 1206. The I/O 
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interface control unit is operatively coupled via an I/O local 
bus 1208 to a storage controller 1210. The storage controller 

5 is operatively coupled to a storage device 1212. Computer 

program instructions 1214 implementing a subtitle server are 
stored on the storage device until the processor retrieves the 
computer program instructions and stores them in the main 
memory. The processor then executes the computer program 

10 instructions stored in the main memory to implement a 
previously described subtitle server to server subtitles 1215, 
stored on the storage device, to a personalized subtitle 
system. 

The subtitle server further includes a network device 

15 1230 coupled to the I/O local bus via a network controller 
1028. The subtitle server uses the network device to access a 
.. communications network and . communicate with personalized 
subtitle systems as previously described. . 

FIG. 5 is a block diagram of subtitle to content 

20 synchronization method wherein the subtitles are associated 
with presented content in accordance with an exemplary 
embodiment of the present invention. In this synchronization 
method, presentation content, such as movie frames 200a, 200b, 
and 200c is associated with subtitles, such as subtitles 202a, 

25 202b, and 202c, stored in the cinema server 110. The 
personalized subtitle system controller is coupled to the 
cinema server via a communications network 114. As the cinema 
server retrieves the presentation content from memory and 
displays the presentation content on a theater screen 104, the 

30 cinema server also serves the associated subtitles to the 
personalized subtitle system controller 108. The personalized 
subtitle system controller receives the subtitles and then 
transmits the subtitles to the HUD device for display to the 
user. As the subtitles are associated with the presentation 

35 content and stored on the cinema server, the subtitles are 
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inherently synchronized to the presentation content. In this 
embodiment, the cinema server only serves subtitles as they 
5 become available while reading and presenting the presentation 

content . 

FIG. 6 is process flow diagram of a personalized subtitle 
display process in accordance with the subtitle to content 
association method of FIG. 5. On start up 301, a personalized 

10 subtitle display process 300 for subtitles associated with 
presentation content waits 3 02 until it is signaled by the 
cinema server that a next subtitle is ready. If the next 
subtitle is ready, the personalized subtitle display process 
receives 3 04 the next subtitle 306 from the cinema server and 

15 generates 308 a subtitle display 310 for presentation to the 
user. If the personalized subtitle display process determines 
312 . .that there are no more subtitles to display, the 
personalized subtitle display process terminates 314. 
Otherwise, the personalized subtitle display process returns 

20 to its, waiting state .302 and waits for the next subtitle to be 
transmitted by the cinema server. 

FIG. 7 is a block diagram of subtitle to content 
synchronization method wherein presented content has an 
associated synchronization signal in accordance with an 

25 exemplary embodiment of the present invention. In this 
synchronization method, the user utilizes the personalized 
subtitle system controller 108 to access a subtitle server 400 
via the communications network 114. The subtitle server 
includes a subtitle database 401 having stored subtitles for a 

30 plurality of presentations such as movies. The user utilizes 
the subtitle server to specify a set of subtitles that are 
read from the subtitle database and stored in the personalized 
subtitle system controller's own subtitle datastore 402. 

In this embodiment, the desired subtitle may not be 

35 available or supported by the cinema server and the movies it 
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is playing. The personalized subtitle system controller can 
connect to a proxy subtitle service via the cinema server; 

5 tell the service what movie the personalized subtitle system 

controller needs a subtitle for, the movie version, what 
language the personalized subtitle system controller needs the 
subtitles in and other options like 'closed caption' , 'hearing 
impaired', (i.e. types of .sub files for a particular 

10 language) etc. 

The subtitle proxy service reads movie media file header, 
or other appropriate data source including user entered data 
to get the 'version 1 of the movie. The proxy service then 
searches the subtitle repositories and retrieves a suitable 

15 subtitle version for the movie version as version information 
for the subtitle file is included in the header or other 
appropriate . data source. The subtitles are . then written (via 
the TCP connection) back to the personalized subtitle system 
controller that stores the subtitles locally. 

20 In one .personalized subtitle system in accordance with an 

exemplary embodiment of the present invention, the above- 
described process is fully automated and occurs without a 
user's awareness that the subtitles were acquired using the 
proxy subtitle service. In this embodiment, the personalized 

25 subtitle system controller notifies the user that the 
subtitles being played are not being broadcast with the source 
media file, i.e. the movie. This is done in case of any error 
correction needed from the user. 

Synchronization between the subtitles stored by the 

30 personalized subtitle system controller and the presentation 
content is provided by a plurality of synchronization signals, 
such as synchronization signals 404a, 404b, and 404c, 
associated with portions of the presentation content, such as 
movie frames of the presentation, such as frames 200a, 200b, 

35 and 200c. The presentation content and synchronization 
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signals are stored in the cinema server 110. As the cinema 
server retrieves the presentation content from memory to 

5 generate the presentation 104 for the user, the cinema server 

also retrieves the associated synchronization signals. The 
cinema server then transmits the synchronization signals to 
the personalized subtitle system controller via the 
communications network. The personalized subtitle system 

10 controller uses the synchronization signals and the previously 
stored subtitles to generate an appropriate subtitle for 
transmission to the HUD device 106 and display to the user. 

The actual format of the synchronization signal may vary. 
For example, in one personalized subtitle system in accordance 

15 with an exemplary embodiment of the present invention, the 
synchronization signal contains no additional information 

., . other than an indication that . the next subtitle is to be 
displayed. In this embodiment, the synchronization signal 
operates as a timing signal used by the personalized subtitle 

20 system controller to time switching to the next subtitle. In 
other personalized subtitle systems in accordance with other 
exemplary embodiments of the present invention, the 
synchronization signal also includes an identifier, such as an 
index number or elapsed time code, of the subtitle that should 

25 be displayed upon receipt of the synchronization signal. In 
these embodiments, the personalized subtitle system controller 
uses the synchronization signal to find the exact subtitle to 
display each time a synchronization signal is received. 

In another personalized subtitle system in accordance 

30 with an exemplary embodiment of the present invention, 
synchronization packets for a movie are transmitted from the 
cinema server to the personalized subtitle system controller 
as a time code encoding the elapsed playing time of the movie. 
FIG. 8 is a block diagram of a subtitle to content 

35 synchronization method wherein the user supplies a 
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synchronization signal in accordance with an exemplary 
embodiment of the present invention. In this embodiment, 

5 there may not be a cinema server. The user can download a 

desired subtitle onto a personalized subtitle system 
controller (for example, while still at home before attending 
the theatre) and bring it to the movie. In this embodiment, 
the user downloads a subtitle file manually onto their 

10 personalized subtitle system controller verifies that the 
version of the film and the version of the subtitle file are 
correctly matched. A personalized subtitle system website may 
facilitate the searching for subtitle files with version 
information for both the subtitle file and the media file to 

15 which the subtitle file is matched with. To use the 
subtitles, the user will manually 'play 1 the subtitle track. 
■ Play ' , x 1 Fast Forward 1 , ' Pause ' , ' Reverse 1 , and - ..' Stop ' 
features are available to the user in the manual mode. 

In slightly more detail, The user utilizes the 

20 personalized subtitle. system controller 108 to access a 
subtitle server 400 via the communications network 114. The 
subtitle server includes a subtitle database 401 having store 
subtitles for a plurality of presentations such as movies. 
The user utilizes the subtitle server to specify a set of 

25 subtitles that are read from the subtitle database and stored 
in the personalized subtitle system controller's own subtitle 
datastore 402 . 

To use the subtitles, the user supplies a synchronization 
signal to the personalized subtitle system controller to input 

30 500 the synchronization signal manually. In response to the 
manually input synchronization signal, the personalized 
subtitle system controller advances to the next subtitle to be 
displayed in sequence. Since the user is viewing the 
presentation content 104 at the same time as the subtitles, 

35 the viewer may increase or decrease the rate at which they 
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supply the synchronization signal in order to advance or 
retard the timing of the transmission of subtitles to the HUD 

5 device 106. 

FIG. 9 is process flow diagram of a personalized subtitle 
display process in accordance with FIG. 7 and FIG. 8. The 
subtitle display process 600 starts (601) by receiving (602) 
subtitles from a subtitle server. The subtitle display 

10 process then waits to receive (604) a synchronization signal, 
such as cinema server synchronization signal 606 or user 
synchronization signal 608, indicating that the subtitle 
display process is to begin displaying subtitles. The type of 
synchronization signal that may be received by the subtitle 

15 display process is dependent upon what type of synchronization 
signals are available, as indicated by the dashed input line 
609,. As previously described, the synchronization signals may. 
be associated with presentation content and transmitted to the 
subtitle display process or may be supplied by the user. If 

20 the synchronization signal .is received (610) the subtitle 
display process selects (611) the subtitle to display and 
displays (612) the subtitle 613 using the previously described 
HUD device. If no synchronization signal is received, the 
subtitle display process continues to wait until a 

25 synchronization signal is received. 

The selection of the next subtitle to display is 
dependent upon the type of synchronization signal sent. If 
the synchronization signal is a timing type signal received 
either from a cinema server or from a user's input, the next 

30 subtitle in a sequence of subtitles is selected for display. 

However, if the synchronization signal contains information 
about the next subtitle to display, the subtitle display 
process uses the synchronization signal to determine which 
subtitle should be displayed. 

35 
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If the subtitle display process determines (614) that 
there are no more subtitles to display, the subtitle display 

5 process stops 616. Otherwise, the subtitle display process 

returns to its waiting mode until another synchronization 
signal is received. 

FIG. 10 is a block diagram depicting using the 
personalized subtitle system with a variety of enhanced 

10 content sources in accordance with an exemplary embodiment of 
the present invention. The personalized subtitle system 
controller 108 may include a short range wireless 
communications link 801, such as a communication link 
employing a Bluetooth protocol, as previously described. As 

15 such, the personalized subtitle system may be used with a 
variety of devices that are also capable of using a short 
range wireless communications link.. These devices may act as 
a subtitle server for serving enhanced content such as 
subtitles to the personalized subtitle system. For example, a 

20 game server 802 may provide enhanced content 800- for a video 
game. The enhanced content is transmitted by the game server 
to the personalized subtitle system controller. The 
personalized subtitle system controller then transmits the 
enhanced content to the HUD device 106 for display to the 

25 user. Other devices may provide enhanced content as well. 

Enhanced content may come from a television display device 
804. For example, a digital TV signal may include a subtitle 
data stream that may contain more information than a typical 
analog captioning signal. In addition, the subtitling 

3 0 information may be combined with a digital TV signal using a 
delayed playback device that stores the TV signal. 

Enhanced content may also come from an electronic book 
display device 806, a digital radio broadcast 808, or an audio 
playback device 810. Other sources of enhanced content may be 

35 accommodated as well. For example, shopping kiosks, DVD 
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players 812, and email display devices may all provide 
enhanced content for display to a user using a personalized 

5 subtitle system. 

In another personalized subtitle system in accordance 
with an exemplary embodiment of the present invention, the HUD 
device includes an audio output device 810, such as an 
earphone, for presentation of audio content to the user. The 

10 enhanced content may then include an audio portion that is 
presented to the user by the personalized subtitle system 
controller using the HUD device's audio output device. 

FIG. 11 is a block diagram depicting using the 
personalized subtitle system at a live event in accordance 

15 with an exemplary embodiment of the present invention. A user 
may use a personalized subtitle system to receive and display 
captioning 900 information for a . live event 901. A 
transcriber 902 or a speech-to-text software program running 
on an automated captioning system 903 observes the live event 

20 and uses a captioning input device 904. to generate captions 
for the live event. A user using a personalized subtitle 
system controller 108 may then access (115) the captions via a 
wireless communications network 114. The personalized, 

subtitle system controller then receives the captions from the 

25 captioning input device via the communications network and 
then transmits the captions to the HUD device for display to 
the user. 

Although this invention has been described in certain 
specific embodiments, many additional modifications and 
30 variations would be apparent to those skilled in the art. It 
is therefore to be understood that this invention may , be 
practiced otherwise than as specifically described. Thus, the 
present embodiments of the invention should be considered in 
all respects as illustrative and not restrictive, the scope of 

35 
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the invention to be determined by any claims supportable by 
this application and the claims' equivalents. 
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